Storage Capacity of a fully - connected parity machine

We study a fully-connected parity machine with K hidden units for continuous weights. The geometrical structure of the weight space of this model is analyzed in terms of the volumes associated with the internal representations of training set. By examining the asymptotic behavior of order parameters in the large K limit, we nd the maximum number c , the storage capacity, of patterns per input unit to be K ln K= ln 2 up to the leading order, which saturates the mathematical bound given by Mitchison and Durbin. Unlike current address 1 the committee machine, the storage capacity per weight remains unchanged compared with the corresponding tree-like architecture.